Sentiment Polarity Classification Research Articles

Meta heuristic approaches for sentiment analysis

AbstractOn the Internet, online microblogging and social networks have experienced tremendous growth. Millions of individuals share opinions on social networking platforms including Twitter, Facebook, YouTube, and Microblogging sites. Millions of individuals express their thoughts on social platforms namely Twitter, Facebook, YouTube, and microblogging sites. As user‐generated content is growing to a huge extent on the internet, the analysis, extraction identification and classification of opinion and sentiment polarity of different aspects has now become a challenging issue in sentiment analysis. In sentiment analysis, aspect level opinion mining, opinion spam review detection, extraction and representation of most relevant accessed data from the user‐generated data is a difficult process. The word vector representation using existing deep learning techniques affects the performance in aspect level opinion mining. Spammers propagate false opinion for payment exchange, which leads to degradation of the financial growth, business, and fame of the organization. To overcome the limitations aspect level opinion mining is used to recognize sentiment and polarity of aspects in the specified text. This work presents various input feature vectors of deep learning approaches aiming to improve performance in aspect level opinion mining. Next, three meta‐heuristic algorithms with k‐means clustering approach for classification of opinion spam reviews on social media transit tweets is presented. The proposed approaches are evaluated and compared with other existing approaches as well as other benchmark datasets namely Restaurant and Amazon reviews.

Senti-Sequence: Learning to Represent Texts for Sentiment Polarity Classification

The sentiment analysis task seeks to categorize opinionated documents as having overall positive or negative opinions. This task is very important to understand unstructured text content generated by users in different domains, such as online and entertainment platforms and social networks. In this paper, we propose a novel method for predicting the overall polarity in texts. First, a new polarity-aware vector representation is automatically built for each document. Then, a bidirectional recurrent neural architecture is designed to identify the emerging polarity. The attained results outperform all of the algorithms found in the literature in the binary polarity classification task.

A Survey on “Sentiment Analysis of E-commerce Website’s Reviews”

Abstract: Sentiment analysis, a vital component of natural language processing, has gained significant relevance in the realm of ecommerce websites. In this digital age, where consumers heavily rely on online reviews to inform their purchase decisions, understanding and harnessing sentiment in ecommerce reviews is paramount. This abstract explores the utilization of sentiment analysis techniques to extract valuable insights from customer feedback, offering a panoramic view of its applications, challenges, and implications. We delve into keyword extraction, sentiment polarity classification, and the integration of sentiment analysis into recommendation systems. This paper also examines the evolving role of sentiment analysis in enhancing user experiences, brand reputation management, and product development. By decoding the sentiments hidden within ecommerce website reviews, businesses can strategically adapt, improve customer satisfaction, and thrive in a highly competitive online marketplace

Research on Emotional Analysis of Online Book Reviews Based on Word2Vec Method

At present, emotional analysis in the field of natural language processing is gradually becoming a popular research direction on social media. With the rise of artificial intelligence technology and deep learning algorithms, word vector representation methods based on neural networks have emerged and emerged in the field of sentiment analysis. According to the current research results of experts, the Word2Vec model has become the mainstream word vector representation method. The algorithm model is based on continuous optimization and iteration of random initialization word vectors, ultimately obtaining a stable performance word vector representation. If the research goal is to obtain sentence vectors, weighting processing is also required. Therefore, after systematically summarizing and analyzing the representation techniques of initialized word vectors and solving sentence vectors through word vector weighting, this article proposes corresponding improvement methods. This article improves the initialization algorithm of its word vector based on the Word2Vec model. On the basis of the original randomly generated word vector, the correlation information of words is added to change the distribution characteristics of the initial word vector, thereby enhancing the performance of the word vector. In the comparative experiment of the user comment dataset, the improved model showed a 1% to 3% improvement in sentiment polarity classification accuracy compared to the other model.

An improved unified domain adversarial category-wise alignment network for unsupervised cross-domain sentiment classification

Cross-domain sentiment analysis (CDSA) aims to learn transferable knowledge from the source domain to facilitate the sentiment polarity classification on the target domain of lacking labeled data. Currently, two types of unsupervised domain adaptation (UDA) methods are widely used in CDSA tasks. One employs the domain adversarial strategy to extract domain-invariant features, and the other utilizes the distance metric strategy to reduce domain distribution discrepancy. However, the fine-grained domain-specific information related to categories aligned between domains is not preserved, which suppresses the performance of target-domain classification. To overcome the mentioned problem, a unified Domain Adversarial Category-wise Alignment Network (DACAN) was proposed in this paper. An integrated network was constructed with progressive multi-level feature learning. Specifically, a feature extraction module was constructed with parameter sharing between two domains at low-level text feature extraction layers. The domain adversarial module was added to enable shared knowledge transfer by extracting domain-invariant information and by updating the shared parameters at the feature extraction layers. A category-wise alignment module was built to achieve local distribution alignment at the high dimension-level semantic layers guided by fine-grained category structure information. Meanwhile, joint constraint was established with domain-invariant constraint based on domain adversarial, and domain-consistency constraint based on category-wise alignment. Comprehensive experiments were conducted on two standard Amazon review datasets. The results show that DACAN outperforms other state-of-the-art UDA methods by 0.7% and 1.1% on the two- and three-category CDSA tasks, respectively. Also, better performance results are achieved with a synergistic UDA scheme compared with a single UDA scheme.

Sentiment Classification Using a Sense Enriched Lexicon-based Approach

The prominent approach in sentiment polarity classification is the Lexicon-based approach which relies on a dictionary to assign a score to subjective words. Most of the existing work use score of the most dominant sense in this process instead of using the contextually appropriate sense. The use of Word Sense Disambiguation (WSD) is less investigated in the sentiment classification tasks. This paper investigates the effect of integrating WSD into a Lexicon-based approach for Sentiment Polarity classification and compares it with the existing Lexicon-based approaches and the state-of-art supervised approaches. The lexicon used in this work is SentiWordNet v2.0. The proposed approach, called Sense Enriched Lexicon-based Approach (SELSA), uses a word sense disambiguation module to identify the correct sense of subjective words. Instead of using the score of the most frequent sense, it uses the score of the contextually appropriate sense only. For the purpose of comparison with the supervised approaches, the authors investigate Naïve Bayes (NB) and Support Vector Machines (SVM) classifiers which tend to perform better in earlier research. The performance of these classifiers is evaluated using Word2vec, Hashing Vectorizer, and bi-gram feature. The best-performing classifier-feature combination is used for comparison. All the evaluations are done on the Movie Review dataset. SELSA achieves an accuracy of 96.25% which is significantly better than the accuracy obtained by SentiWordNet-based approach without WSD on the same dataset. The performance of the proposed algorithm is also compared with the best-performing supervised classifier investigated in this work and earlier reported works on the same dataset. The results reveal that the SVM classifier performs better than SentiWordNet approach without WSD. However, after incorporating WSD the performance of the proposed Lexicon-based approach is significantly improved and it surpasses the best-performing supervised classifier (SVM with bi-gram features).

An Efficient Deep Learning for Thai Sentiment Analysis

The number of reviews from customers on travel websites and platforms is quickly increasing. They provide people with the ability to write reviews about their experience with respect to service quality, location, room, and cleanliness, thereby helping others before booking hotels. Many people fail to consider hotel bookings because the numerous reviews take a long time to read, and many are in a non-native language. Thus, hotel businesses need an efficient process to analyze and categorize the polarity of reviews as positive, negative, or neutral. In particular, low-resource languages such as Thai have greater limitations in terms of resources to classify sentiment polarity. In this paper, a sentiment analysis method is proposed for Thai sentiment classification in the hotel domain. Firstly, the Word2Vec technique (the continuous bag-of-words (CBOW) and skip-gram approaches) was applied to create word embeddings of different vector dimensions. Secondly, each word embedding model was combined with deep learning (DL) models to observe the impact of each word vector dimension result. We compared the performance of nine DL models (CNN, LSTM, Bi-LSTM, GRU, Bi-GRU, CNN-LSTM, CNN-BiLSTM, CNN-GRU, and CNN-BiGRU) with different numbers of layers to evaluate their performance in polarity classification. The dataset was classified using the FastText and BERT pre-trained models to carry out the sentiment polarity classification. Finally, our experimental results show that the WangchanBERTa model slightly improved the accuracy, producing a value of 0.9225, and the skip-gram and CNN model combination outperformed other DL models, reaching an accuracy of 0.9170. From the experiments, we found that the word vector dimensions, hyperparameter values, and the number of layers of the DL models affected the performance of sentiment classification. Our research provides guidance for setting suitable hyperparameter values to improve the accuracy of sentiment classification for the Thai language in the hotel domain.

Rule Based Fuzzy Computing Approach on Self-Supervised Sentiment Polarity Classification with Word Sense Disambiguation in Machine Translation for Hindi Language

With increasing globalization, communication among people of diverse cultural backgrounds is also taking place to a very large extent in the present era. Issues like language diversity in various parts of the world can lead to hindrance in communication. The usage of social media and user-generated material has grown at an exponential rate and existing supervised sentiment polarity classification techniques need labelling for the training dataset. In this study, two problems have been analyzed. First, sentiment analysis of the Twitter dataset and sense disambiguation of morphologically rich Hindi language. A rule-based fuzzy logics-based system for self-supervised sentiment classification was used to compute and analyze the self-supervised or completely unsupervised sentiment categorization of a social-media dataset using three types of lexicons.  The combination of fuzzy with three different types of lexicons gives sentiment analysis a new path. The unsupervised fuzzy rules integrate the fuzziness of both negative as well as positive scores, and fuzzy logic-based systems can cope with ambiguity and vagueness. The fuzzy-system uses an unsupervised/self-supervised fuzzy rule-based technique to identify text using natural language processing (NLP) and sense of word. We compared the results of fuzzy rule based self-supervised sentiment classification by using three types of lexicons on five different datasets, with unsupervised as well as supervised sentiment classification techniques. Second, using cross-lingual sense embedding rather than cross-lingual word embedding resolves the ambiguity issue. The word sense embeddings are produced for the source languages to learn multiple or various senses of the words. Different evaluation metrics depict an improved performance for English-Hindi language.

ERF-XGB: Ensemble Random Forest-Based XG Boost for Accurate Prediction and Classification of E-Commerce Product Review

Recently, the concept of e-commerce product review evaluation has become a research topic of significant interest in sentiment analysis. The sentiment polarity estimation of product reviews is a great way to obtain a buyer’s opinion on products. It offers significant advantages for online shopping customers to evaluate the service and product qualities of the purchased products. However, the issues related to polysemy, disambiguation, and word dimension mapping create prediction problems in analyzing online reviews. In order to address such issues and enhance the sentiment polarity classification, this paper proposes a new sentiment analysis model, the Ensemble Random Forest-based XG boost (ERF-XGB) approach, for the accurate binary classification of online e-commerce product review sentiments. Two different Internet Movie Database (IMDB) datasets and the Chinese Emotional Corpus (ChnSentiCorp) dataset are used for estimating online reviews. First, the datasets are preprocessed through tokenization, lemmatization, and stemming operations. The Harris hawk optimization (HHO) algorithm selects two datasets’ corresponding features. Finally, the sentiments from online reviews are classified into positive and negative categories regarding the proposed ERF-XGB approach. Hyperparameter tuning is used to find the optimal parameter values that improve the performance of the proposed ERF-XGB algorithm. The performance of the proposed ERF-XGB approach is analyzed using evaluation indicators, namely accuracy, recall, precision, and F1-score, for different existing approaches. Compared with the existing method, the proposed ERF-XGB approach effectively predicts sentiments of online product reviews with an accuracy rate of about 98.7% for the ChnSentiCorp dataset and 98.2% for the IMDB dataset.

Arabic aspect sentiment polarity classification using BERT

Aspect-based sentiment analysis (ABSA) is a textual analysis methodology that defines the polarity of opinions on certain aspects related to specific targets. The majority of research on ABSA is in English, with a small amount of work available in Arabic. Most previous Arabic research has relied on deep learning models that depend primarily on context-independent word embeddings (e.g. word2vec), where each word has a fixed representation independent of its context. This article explores the modeling capabilities of contextual embeddings from pre-trained language models, such as BERT, and making use of sentence pair input on Arabic aspect sentiment polarity classification task. In particular, we develop a simple but effective BERT-based neural baseline to handle this task. Our BERT architecture with a simple linear classification layer surpassed the state-of-the-art works, according to the experimental results on three different Arabic datasets. Achieving an accuracy of 89.51% on the Arabic hotel reviews dataset, 73.23% on the Human annotated book reviews dataset, and 85.73% on the Arabic news dataset.

A disentangled linguistic graph model for explainable aspect-based sentiment analysis

Aspect-based sentiment analysis (ABSA) aims to use interactions between aspect terms and their contexts to predict sentiment polarity for given aspects in sentences. Current mainstream approaches use deep neural networks (DNNs) combined with additional linguistic information to improve performance. DNN-based methods, however, lack explanation and transparency to support predictions, and no existing model completely solves the trade-off between explainability and performance. In contrast, most previous studies explain the relationship between input and output by attribution; however, this approach is insufficient to mine hidden semantics from abstract features. To overcome the aforementioned limitations, we propose a disentangled linguistic graph model (DLGM) to enhance transparency and performance by guiding the signal flow. First, we propose a disentangled linguistic representation learning module that extracts a specific linguistic property via neurons to help capture finer feature representations. To further boost explainability, we propose a supervised disentangling module, in which labeled linguistic data help reduce information redundancy. Finally, a cross-linguistic routing mechanism is introduced into the signal propagation of linguistic chunks to overcome the defect of distilling information in an intralinguistic property. Quantitative and qualitative experiments verify the effectiveness and superiority of the proposed DLGM in sentiment polarity classification and explainability.

Justifying Arabic Text Sentiment Analysis Using Explainable AI (XAI): LASIK Surgeries Case Study

With the increasing use of machine learning across various fields to address several aims and goals, the complexity of the ML and Deep Learning (DL) approaches used to provide solutions has also increased. In the last few years, Explainable AI (XAI) methods to further justify and interpret deep learning models have been introduced across several domains and fields. While most papers have applied XAI to English and other Latin-based languages, this paper aims to explain attention-based long short-term memory (LSTM) results across Arabic Sentiment Analysis (ASA), which is considered an uncharted area in previous research. With the use of Local Interpretable Model-agnostic Explanation (LIME), we intend to further justify and demonstrate how the LSTM leads to the prediction of sentiment polarity within ASA in domain-specific Arabic texts regarding medical insights on LASIK surgery across Twitter users. In our research, the LSTM reached an accuracy of 79.1% on the proposed data set. Throughout the representation of sentiments using LIME, it demonstrated accurate results regarding how specific words contributed to the overall sentiment polarity classification. Furthermore, we compared the word count with the probability weights given across the examples, in order to further validate the LIME results in the context of ASA.

Language interpretation in travel guidance platform: Text mining and sentiment analysis of TripAdvisor reviews.

The opinions and feelings expressed by tourists in their reviews intuitively represent tourists' evaluation of travel destinations with distinct tones and strong emotions. Both consumers and product/service providers need help understanding and navigating the resulting information spaces, which are vast and dynamic. Traditional sentiment analysis is mostly based on statistics, which can analyze the sentiment of a large number of texts. However, it is difficult to classify the overall sentiment of a text, and the context-independent nature limits their representative power in a rich context, hurting performance in Natural Language Processing (NLP) tasks. This work proposes an aspect-based sentiment analysis model by extracting aspect-category and corresponding sentiment polarity from tourists' reviews, based on the Bidirectional Encoder Representation from Transformers (BERT) model. First, we design a text enhancement strategy which utilizes iterative translation across multiple languages, to generate a dataset of 4,000 reviews by extending a dataset of 2,000 online reviews on 1,000 tourist attractions. Then, the enhanced dataset is reorganized into 10 classifications by the Term Frequency-Inverse Document Frequency (TF-IDF) method. Finally, the aspect-based sentiment analysis is performed on the enhanced dataset, and the obtained sentiment polarity classification and prediction of the tourism review data make the expectations and appeals in tourists' language available. The experimental study generates generic and personalized recommendations for users based on the emotions in the language and helps merchants achieve more effective service and product upgrades.

Evaluating semantic representations for extended association rules

In this work, we evaluate the impact of changing the semantic text representation on the performance of the AR-SVS (extended association rules in semantic vector spaces) algorithm on the sentiment polarity classification task on a paper reviews dataset. To do this, we use natural language processing techniques in conjunction with machine learning classifiers. In particular, we report the classification performance using the F1 and accuracy metrics. The semantic representations that we used in our evaluation were chosen based on a systematic literature review, leading to an evaluation of AR-SVS with FastText, GloVe, and LDA2vec representations, with word2vec providing the baseline performance. The results of the experiments indicate that the choice of semantic text representation does not have major effects on the performance of AR-SVS for polarity classification. Furthermore, the results resemble those obtained in the original AR-SVS article, both in quantitative and qualitative terms. Thus, while direct improvements in classification performance were not found, we discuss other aspects and advantages of using different semantic representations.

Context-Aware Sentiment Analysis using Tweet Expansion Method

The large source of information space produced by the plethora of social media platforms in general and microblogging in particular has spawned a slew of new applications and prompted the rise and expansion of sentiment analysis research. We propose a sentiment analysis technique that identifies the main parts to describe tweet intent and also enriches them with relevant words, phrases, or even inferred variables. We followed a state-of-the-art hybrid deep learning model to combine Convolutional Neural Network (CNN) and the Long Short-Term Memory network (LSTM) to classify tweet data based on their polarity. To preserve the latent relationships between tweet terms and their expanded representation, sentence encoding and contextualized word embeddings are utilized. To investigate the performance of tweet embeddings on the sentiment analysis task, we tested several context-free models (Word2Vec, Sentence2Vec, Glove, and FastText), a dynamic embedding model (BERT), deep contextualized word representations (ELMo), and an entity-based model (Wikipedia). The proposed method and results prove that text enrichment improves the accuracy of sentiment polarity classification with a notable percentage.

Attention-based aspect sentiment classification using enhanced learning through cnn-Bilstm networks

Deep neural networks (dnn) techniques for aspect-based sentiment classification have been widely studied. The success of these methods depends largely on training data which are often inadequate because of the rigor involved in manually tagging large collection of opinionated texts. Attempts have been made to transfer knowledge from document-level to aspect-level sentiment task. However, the success of this approach is also dependent on the model because aspect sentiment data like other type of texts contain complex semantic features. In this paper, we present an attention-based deep learning technique which jointly learns on document and aspect-level sentiment data and which also transfers learning from the document-level data to aspect-level sentiment classification. It basically consists of a convolutional layer and a bidirectional long short-term memory (Bilstm) layer. The first variant of our technique uses convolutional neural network (cnn) to extract high-level semantic features. The output of the feature extraction is then fed into the Bilstm layer which captures the contextual feature representation of the texts. The second variant applies the Bilstm layer directly on the input data. In both variants, the output hidden representation is passed to an output layer using softmax activation function for sentiment polarity classification. We evaluate our model on four standard benchmark datasets which shows the effectiveness of our approach with improvements over baselines. We also conduct ablation studies to show the effect of the different document-level weights on the learning techniques.

Policy-focused Stance Detection in Parliamentary Debate Speeches

Legislative debate transcripts provide citizens with information about the activities of their elected representatives, but are difficult for people to process. We propose the novel task of policy-focused stance detection, in which both the policy proposals under debate and the position of the speakers towards those proposals are identified. We adapt a previously existing dataset to include manual annotations of policy preferences, an established schema from political science.  We evaluate a range of approaches to the automatic classification of policy preferences and speech sentiment polarity, including transformer-based text representations and a multi-task learning paradigm. We find that it is possible to identify the policies under discussion using features derived from the speeches, and that incorporating motion-dependent debate modelling, previously used to classify speech sentiment, also improves performance in the classification of policy preferences. We analyse the output of the best performing system, finding that discriminating features for the task are highly domain-specific, and that speeches that address policy preferences proposed by members of the same party can be among the most difficult to predict.

Integrating Character-level and Word-level Representation for Affect in Arabic Tweets

Affect tasks, which range from sentiment polarity classification to finer grained sentiment strength and emotional intensity detection, have become of increasing interest due to the vast amount of user-generated content and advanced learning models. Word representation models have been leveraged effectively within a variety of natural language processing tasks. However, these models are not always effective in the context of social media. When dealing with social media posts in Arabic, the use of Arabic dialects needs to be considered. Although using informal text to train word-level models can lead to the identification of words that convey the same meaning, these models are unable to capture the full extent of the words that are used in the real world due to out-of-vocabulary (OOV) words. The inability to identify such words is one of the main limitations of word-level models. One approach of overcoming OOV is through the use of character-level embeddings as they can effectively learn the vectors of word parts or character n-grams. This study uses a combination of character-level and word-level models to identify the most effective methods by which affective Arabic words in tweets can be represented semantically and morphologically. We evaluate our generated models and the proposed method by integrating them in a supervised learning framework that was used for a range of affect tasks and other related tasks. Our findings reveal that the developed models surpassed the performance of state-of-the-art Arabic pre-trained word embeddings over eight datasets. In addition, our models enhance previous state-of-the-art outcomes on tasks involving Arabic emotion intensity, outperforming the top-systems that used advanced ensemble learning models and several additional features.

Opinion Triplet Extraction for Aspect-Based Sentiment Analysis Using Co-Extraction Approach

In aspect-based sentiment analysis, tasks are diverse and consist of aspect term extraction, aspect categorization, opinion term extraction, sentiment polarity classification, and relation extractions of aspect and opinion terms. These tasks are generally carried out sequentially using more than one model. However, this approach is inefficient and likely to reduce the model’s performance due to cumulative errors in previous processes. The co-extraction approach with Dual crOss-sharEd RNN (DOER) and span-based multitask acquired better performance than the pipelined approaches in English review data. Therefore, this research focuses on adapting the co-extraction approach where the extraction of aspect terms, opinion terms, and sentiment polarity are conducted simultaneously from review texts. The co-extraction approach was adapted by modifying the original frameworks to perform unhandled subtask to get the opinion triplet. Furthermore, the output layer on these frameworks was modified and trained using a collection of Indonesian-language hotel reviews. The adaptation was conducted by testing the output layer topology for aspect and opinion term extraction as well as variations in the type of recurrent neural network cells and model hyperparameters used, and then analysing the results to obtain a conclusion. The two proposed frameworks were able to carry out opinion triplet extraction and achieve decent performance. The DOER framework achieves better performance than the baselines on aspect and opinion term extraction tasks.

An Improved Model for Analyzing Textual Sentiment Based on a Deep Neural Network Using Multi-Head Attention Mechanism

Due to the increasing growth of social media content on websites such as Twitter and Facebook, analyzing textual sentiment has become a challenging task. Therefore, many studies have focused on textual sentiment analysis. Recently, deep learning models, such as convolutional neural networks and long short-term memory, have achieved promising performance in sentiment analysis. These models have proven their ability to cope with the arbitrary length of sequences. However, when they are used in the feature extraction layer, the feature distance is highly dimensional, the text data are sparse, and they assign equal importance to various features. To address these issues, we propose a hybrid model that combines a deep neural network with a multi-head attention mechanism (DNN–MHAT). In the DNN–MHAT model, we first design an improved deep neural network to capture the text’s actual context and extract the local features of position invariants by combining recurrent bidirectional long short-term memory units (Bi-LSTM) with a convolutional neural network (CNN). Second, we present a multi-head attention mechanism to capture the words in the text that are significantly related to long space and encoding dependencies, which adds a different focus to the information outputted from the hidden layers of BiLSTM. Finally, a global average pooling is applied for transforming the vector into a high-level sentiment representation to avoid model overfitting, and a sigmoid classifier is applied to carry out the sentiment polarity classification of texts. The DNN–MHAT model is tested on four reviews and two Twitter datasets. The results of the experiments illustrate the effectiveness of the DNN–MHAT model, which achieved excellent performance compared to the state-of-the-art baseline methods based on short tweets and long reviews.